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Por the past £xfW years a group of doctors at the 
University of Hisconsin has hean developing a realistic 
computer-based simulation of the patient'-^physician encounter* 
Dtilizing the time^shared computers, large memory and high speed 
logic^ they have developed a highly sophisticated program Mhich 
permits the physician to work-up and diagnose patients in the 
in-patient and out--patient environments This program has been tested 
in over 1,000 simulated encouuters and is currently operational at a 
number of medical centers around the country. Criteria for standard 
evaluations have been developed and tested* The main drawback is that 
the physician must communicate with the computer via a relatively 
expensive computer teriainal. In order to overcome the necessity of 
using expensive computer terminals they propose to oombine their 
fully operational simulator with a Voice-Eesponse System* Combining 
rhese two systems would permit a physician to take a simulated 
encounter using any standard telephone (touchstone or dial)* The 
physician would communicate with the cop^uter by dialing in the 
necessary test reguestj. diagnosis, etc. The computer would 
communicate with th# physician through easily understood verbal 
responses* This interaction could take place anywhere there is a 
telc^phone and literally thousands of encounters would he possible 
simultaneously* (MV) 
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Purposes 



A computerized voice response system has been developed and tested 
during Che past year Involving groups of medica! students, medical residents 
and practicing physicians. Three general purposes were estabUshed for the 
project. They were* 

1 ) to davelop a prototype computerized voice system adapted to the 
needs of conitlnuing medical education 

2) to field test the system using volunteer-practicing physicians 
and student physicians 

3) to provide evidence that a computerized voice is a sound method 
of communication which can assist In a madlcal self -^evaluation 
educatiOTi program. 

The Voice System 

The value of a computartzed voice system fs to extend computer usage 
via presentlyex^stlngi end/or ineKpensive technology, A major limitation 
of computer applications has been the requirement of a direct link to the 
computer via expensive terminal equipment. This usually requires either 
direct irtterfsce of monitoring equipment with expensive modems and 
cables or use of eUher *hard wired" or 'acoustically' coupled computer termi 
nals. (Host medrcal applications have utinzed computer terminals 
connected to tlhe main co^nputer by acoustic couplers.) These terminals 
cost between $1,500 and $10,000 each and require the user to be In a 
specific location, i.e., reducing accessibility, and to be able to use a key 
board, r.e*i reducing versatnity. 

Presently, voice system users require onliy a telephone and are pro* 
vided wfth a touchtone pad, amplifier and Instruction manual. Physician- 
users communicate with the computer with a standard telephone (touchtone 
or dial),, diaiing the necessary test, request, diagnoses, etc. IhlB 
computer Interaction can take place anywhere there is access to a telephone 
and Is not dependent upofi famntarity with a teminal keyboard. 

The system itself consists of a liirge and a small computer, a Votrax 
Voice generating wnit and a TEAC tape recorder. A diagram iHustratlng 
the system cafi be found in Appendix 1. Following^ (see Appendix 2) > 5s 
a diagram sho:^ing the appropr fate set-up of this equipment. A copy of the 
user Instruction manual has been Included with this report* 



A. Hardware 

The following Is a description of the system, A vol ca generating unit 
which is a commercially developed VotraK synthesizing unit is connected to 
a small cOTputer (POP-8) and together they produce a simulation of the human 
braln-^vocal system. That is, under program control » the computer (brain) 
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and voice synthesiser create the spoken English words* ^ These words are 
formed using a computer terminal and each must be made individually u&mg 
any combinatmn of the avanable phonemes* The VotraK unit Is capable of 
producing 6^ phonemes. The voice system voeafaulary now consists of some 
3,500 words. The words are stored on the large computer |PDP-15)- The 
voice system uses a termina! tfiat is either a touchtone telephone or a 
touchtone pad which can be attached to a standard dial telephone. 

The system itself was put together and is mainitalned by an slecitrtcal 
engineer/ AH programming was done in HI IS (Hf»S is a programmr^r languaS^ 
v/hich is a dialect of MUMPS) by a computer programmer. Several students 
were hircid on a part'- time basis to build the words over the past year. 
Great paJiis were taken to work and rework each word untif it soynded 
satisfactory. AIS the work was done between June of 1975 and June of 1976* 
with the exception of a JtmHed amount of vocabulary buMdlng and preliminary 
design of the system. The voice system can be used anywhere in the country- 



B. Software 

The voice system is designed to simulate the patient=physisian encounter. 
Five scripts are available on the system and each script involves a patient 
dtsord€r for which a physician should do a work-up and arrive at a correct 
diagnosis. Below is a brief description of each scr'^pt alortg with the 
correct diagnosis; 

CORRECT DIAGNQSiS 

V/aldenstrom's 
macroglubul inemia 

Normal patient 



CASE # 
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SCRIPT 

- A gz^year^old man with weakriess, chM'ls and 
fever. 

A SS-^year^old man with a possible 
myoca rd i a 1 i n f a ret i on . 

- A B^-year'-old man with cough and weight 
loss . 

- A A3^year"0ld man brought to the hospital 
after col laps i ng * 

- A 36-year-^old female with persistent back 
pa in * 

There are 368 tests which may be ordered: and results are returned as 
they would be In a real hospital situation. When a test is ordered, the 
physician fs automatically charged for it and at the concJuslon of the 
work-up, all expenses are totaled and the user is presented with a bill. The 
physician chooses a script. Is presented with a pat ient compla mt, orders 
tests, receives results and makes a diagnosis. 
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Tuberculosi s 



Aortic aneurysm 



Lupus erythematosus 
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All tests and diagnoses are 



Sh^ system not only produces synthesized voice but is also capable of using 
natural vole ' nterfacing a TEAC tape recorder with the system. An 
i-n^" application using the »^natura1" voice Is described at the end of this report. 
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coded; and all input is achieved by pressing the touchtone keys, 

C . Parameters of Evaluation 

At the conclusion of the i/ork^up the user is grven the opportunity 
to have his/her work^up evaluated* Four of the scripts include evaluative 
measures utiliziing a model developed by the Medfcal Education Group at the 
Univ. of Illinois* This rnodel uses a scoring systam ranging from +3 to '3, 
Using these parameters^ the voice project evalgatiori committee assigned each 
test for a particular script a value within the t3 to ^^3 range.* Thus, tests 
which were considered necessary to perform ranged^^^^ward the positive end of 
the scale while tests which were important to avoid rang?ff°^CoWaTd the 
negative pole of the scale* 

+3 essential to do 
+2 important to do 
+1 helpful 

0 optional 
"1 useless 
"2 harmful 

"-3 essential to avoid 



Scores of efflctencyg proficiency^ errors of omission, errors of commls* 
sion and a competence index were calculated from these values using the 
following formulas; 



Ef f iclency 



number of positive items selected 

It m — ^ ----- ■ -J. - ^--.^^.^^ ^ 

total number of items selected 



Proficiency 



weighted (algebraic) sum of items select ed 
maximum possible score 



P , , weig hted sum or positive stems not selected 

Errors qf omission - t . , , — — = — — — " " -- - -- — ^ 

maximum possible score 

e ^ . . weighted sum of negative items selected 

Errors of cormii ss ion ^ ?_ — — r^ t - — - - ^ 

maximum possible score 

^ __ * * combined function of efficiency 
etence index - " ^* * / 

and proficiency 



Since no physician would be expected to order every test with a 
positive value, no participant would be expected to obtain a perfect (lOO) 
score. For each ca^e a competence Index between 30 and ^0 was projected 
as normil and a score of above kO^ superior* A competence Index iibove 
60 would, in most cases, occur only If one undertook an excessively 
complete work^up. 

At the end of the scriipts> the computer asks if one wishes to have the 
work-up evaluated. A 'yes' resporiSe gives the following Informations 



Efficiency 
Prof 1 c iency 
Errors of Omission 
Errors of Commission 
Competencri Index 



Number of days In hospital 
Cost of diagnostic work^^up 
Cost of hospital room ($90/day) 
Total cost of hospf ta 1 izat ion 
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All data is stored In the computer and cm be retrieved at any time. 
This data includes the length of time the user was on the systerpt tests 
ordered, times results were given, correct of Incorrect diagnosis and usurU 
scores. An example of the data output summary can be found In Appendix 3^ 

D . Chronology of Events 

During the initial period of the project, from July 1975 to December 
1975* the work centered on transferring the simulation encounter program 
from the Univac 1110 computer to the PDP-15. In addition, the Votrax voice 
synthesizing unit was obtained and made operatronal by interfacing jt with 
a PDP-8 computer* Finally, the process of buildtng the necessary dictionary 
words and retrieve the words from the PDP-15 as part of the simulated 
patient-^physiclan encounter was carried out during this period. 

In January, 1976, the Department of Continuing Medical Education at 
the University of Wisconsin was contacted to coordinate pract f clng-^physlclan 
use of the simulation encounter. Physicians from around the state were 
contacted and sent equipment (touchtone pad, amplifier, and Instruction 
nianual) so they could participate In the study. At the same time, extensive 
system testing and debugging was carried out by the project personnel. 

Beginning in March, 1976 the testing of the Voice System was carried 
out. Although the system had responded well when tested by project personnel, 
the variations In the quality of phone lines around the state caused numerous 
difficulties. After extensive debugging and eKper imentation with alternatives 
(e.g., hooking the touchtone pad to the phone via connecting wires and clips), 
the system still did not respond very consistently. For example, over 50 
physicians were sent equipment, yet less than half of these were able to get 
connected to the voice system and work-up one or more cases. The results 
Indicated that although the technology for a voice sys»tem Is adequate, the 
necessary phone line technology is not yet existent on a statewide basis. 

Fiel d Testing 

A* Student Physicians 

The system became operational In February of 1976.' At that time 
volunteers were solicited to test the system. These volunteers consisted of 
third year medical students and first, second and third year medUal residents 
at the University of Wisconsin Medical Center. They were asked to spend 
from three to four hours testing the system. They Identified ''bugs" In the 
system which were then corrected. The students were provided with tasting 
units where each unit consisted of a touchtone pad, instructifjnal manual 
and amplifier. The amplifier eliminated the necessity of holding the tele- 
phone to the ear, thus, the individual enjoyed free use of his hands for 
note taking during the work-up. The students were compensated for their 
assistance with a medical text. 

B, Practicing Physicians 

Meanwhile, the Postgraduate Education Department (PED) at the University 
of Wisconsin Medical Center solicited volunteer practicing physicians from 
through^t the state to test the system. This group deals with physicians 
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throughout the state on a continuing medical education basis, hence, their 
recruiting abinty was of great value to us. An agreement with PED to 
arrange for testing was made when this project was conceived* They were 
contracted to recruit physicians^ to arrange testing and to distribute the 
necessary equipment. Contact v/ith the physicians v-zas made primarily by 
mail, along with announcements over an educational telephone netv/ork. 

In the first part of Aprils the PED was provided with ten testing units* 
Fifty physicians had been recruited. For participating In the study, the 
physicians were offered the choice of a medical text or 3n honorarium* PED 
asked these physicians to spend a minimum of two hours working v/ith the 
system in an effort to complete as many of the five scripts as possible* 
Many spent more than two hours. To insure anonymityj each user was assigned 
a number by PED; hence, all data was stored In the computer under a number 
rather than a name. 

Host physicians were not v/ithin the Madison area codaj and therefore, 
a long distance call was necessary* The physicians called the University 
of Wisconsin Centrex operator on the Watts line and were then connected to 
the computer. This made the calls inexpensive and In addition the doctors 
were not billed for telephone usage. 

C. Testing Diffic ulties 

Throughout the course of testing^ mechanical difficulties were a 
constant problem. Users would oTder tests and ones different from those 
that were ordered would be returned. Ac times the program would stop In 
mid-^script requiring the user to begin the case anew. It was discoverGd 
through extensive testing that the source of these dlfftcultles was either 
in the telephone or the touchtone pad of a combination of the two. At 
times the touchtone pads did not send out sufficiently sharp frequencies, 
so that test order confusion resulted. Also, some telephone lines did not 
consistently relay the touchtone signal s^ even when there were sufficient 
frequencies. 

These difficulties were thus dependent upon touchtone quality and upon 
location of the telephone being used. The problams were transient and 
unpredictable In terms of establishing a priorf which locations would 
provide suitable phone line tranimr ss Ion , or identifying a faulty touchtone 
pad . 

No problems were encountered when working with the system using a 
touchtone telephone. Unfortunately ^ most of the physicians did not have 
touchtone phones. 

F i n d i n j 

A, Affective Dimension^ Student Physicians 

Upon completion of their work^^ups, student physicians were asked to 
complete a questionnaire aimed at measuring reactions to the voice computer. 
The following questions were asked: 



Were you able to understand the speech? 
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Did you find this problem-solving exercise realistic? 
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Do you feal this Is a reasonable mathod of communication? 
Do you feel this could serve a continuing education function? 
Do you feel this Is an effective means of self^evaluation? 
Did you encounter mechanical difficulty? 
Did you enjoy working with the system? 

Since the sample group of users was smaM in each case (7 3rd-year 
madical students and 1^ medical residents). It was not germane to perform 
a statistical analysis of the responses to these questions. Howevers the 
responses indicated that none of the students eKperierieed difficulty in 
understanding the computer speech. Answers to other questions provided 
less than unanimous results: 

Student Physician Questionnaire Results^ 

Were you able to understand the speech? 0 

Did you find this problem'-soiving exercise realistic? 6 

Do you feel this Is a reasonable method of 

communication/ • ^ 

Do you feel this could serve a continuing education 
function? ^ 

Do you feel this is an effective means of sel f^evaluatlon?? 

Did you encounter mechanical difficulties? 0 

Did you enjoy v/orking with the system? 0 

WL - within nmits (responses that were somewhere between a definite yes or no) 

It is, for example, somewhat ambiguous as to whether users felt the 
system to be a realistic eKperlence In problem solving; a majority of users 
agreed that It was realistic, but expressed reservations because the system 
concentrated on laboratory diagnosis and did not allow patient management 
nor did It allow for a sense of the patient, A majority Judged it to be 
a reasonable means of communication, while a smaller and opposed majority 
agreed it could serve a continuing medical education function. Users 
were guarded In their opinion that the voice system offers an effective 
means of self-evaluation. Several expressed the opinion that It left out 
pat rent^physician interaction; some did not agree with the evaluation; . 
others suggested that It measured how well one -'played the game," A 
number of users encountered mechanical difficulty, but most agreed that 
the system itself was enjoyable to use. 



yes no 

21 0 

8 7 

18 1 

16 5 

10 U 

16 5 

12 3 



EKLC 



breakdown by year-of -^tra I n tng of these questionnaire results Is given In 
Appendix ^l, Tfole One. 
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B. Affect tve Djmens ion - Pract i c i ng Phys i cl ans 

Upon completion of the work^up the practicing physicians were also 
asked to complete a questionnaire on their reactions to the voice computer. 
The following questions were asked^ 

Did the problems closely approximate real llfej (i.e.j were they realistic?) 

Old you learn something from the simulated problems, ( 1 ,e. , is this a 
good se 1 f =eva 1 ua t Ion method?) 

Do you know more about your strengths and weaknesses, (Ke,, is this 
a good sel f -eval uat f on method?) 

Did you enjoy working through the patient problems? 

Did you encounter mechanical or technical difficulties? 

Were the instructions easy to follow? 

How would you rate the understandabfVity of the voices 
Exce 1 lent G ood Acceptable B ad Unacceptable_ 

Practicing Physician Questionnaire Re sults 

jWL yes no 

Did the problems closely approKimate real life? 19 1 

Did you learn something from the simulated problems? 4 15 2 

Do you know more about your strengths and weaknesses? 1 14 5 

Did you enjoy working through the patient probleniS? - 20 2 

Did you encounter mechanical or technical difficulties? 1 19 20 

Were the Instructions easy to follow? 4 15 3 

Ratings on the Understandabl 1 i ty of the Voice 
EKcellent 

Good 3 

Acceptabl e 16 

Bad 3 

Unacceptabl e 1 

•«WL ^ within limits (responses that were somewhere between a definite yes or no) 

9 out of 30 practicing physicians returning questionnaires were unable to 
get on the Voice Response due to telephone connection difficulties* 
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The results indicate that although the majority of practicing physicians 
expGrienced some form of difficulty In using the Voice Response system this 
same large majority enjoyed v^orking v/ith the system and felt they learned 
something from using it. All in all the responses indicate a generally 
positive attitude by the practicing physician towards using the voice system 
and in terms of their ability to understand the actual voice. 

C. Voice Response Results 

For each script used by a physician, data on the time spent per case, ©n 
the cost of the work^up (including both hospital charges and charges for 
tests orderd) bnd on whether a correct diagnosis was obtained, was collected. 
In addition, for those four scripts (6, 9, 10 and ^i5) that had the scoririg 
system of essential to nonessential testSj data on the efficiency^ proficiencyi 
errors of omissiDn, errors of commission, and competency Index-scores were 
collected* Tables One, Two, and Three summarize the data. 

These tables contain the means^ standard deviations, and n's for each 
of the parameter scores for each case as well as the combined total for all 
cases. Table One presents the data for the medical students, Table Two 
for the resident physicians, and Table Three for the practicing physicians. 

The data In these tables general ly indl cate no particular differences or 
trends dye to level of eKperlence; that is, practicing physicians do not 
consistently score more positively on the parameters of evaluation than 
either the resident physicians or medical students. This however Is most 
probably due to the content level of the scripts and the nature of the 
simulation encounter. None of the scripts and associated tests require any ^ 
degree of knowledge specialization that comes with greater medical train- 
Ing and/or experience. Furtherj the simulation encounter does not require 
extensive patient management ski Ms of directing and modifying t^ierapeutlc 
interventions on the basis of patient response. Rather it concentrates on 
obtaining and Interpreting diagnostic data for which it could be assumed 
all three levels of physicians are proficient. 

Interestingly, one parameter, the overall diagnostic rate (l,e., the 
mean correct diagnosis by physician level for all patient cases tested) 
goes down for the practicing physician. The rates are 77lf 79% and 6l% for 
medical students^ resident physicians and practicing physicians, respect^ 
ively. This proposition could be empirically substantiated through testing 
for therapeutic intervention parameters on these three subgroups. Such a 
feature Is provided in a study presently underway. 

To summarize the data in Tables One, Two and Three (I.e., see Table 
Four) the threes parameters of a) time spent on a cases b) total cost in-- 
eluding both work-^up and hospital charges, and c) competency index rating 
were considered* It was felt that these parameters, In addition to the 
Correct/ Incorrect diagnosis variable, best measured differing aspects of 
the simulation. Since Case 13 did not contain the expected analysis of the 
tests (I.e., efficiency to competency^ Index scores) It was not Included In 
this summary. Also, in some infitances mechanical difficulties led to in- 
complete work^up of cases which did not include these scoring parameters. 
Consequently, the number of subjects for this summary is 20, 29 and 36 for 
the medical students, the resident physicians, and the practicing physi^ 
clans, respectively. 
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Table Four presents the means, standard deviations and N^s for the 
summarJ-zat ion using the parameters of time, cost and competency IndeK for^, 
each physician level tested. In this summary the level of physician train 
ing (e.g., medical students vs , residents and/or practicing physician) 
d©es show percentage increases for all three measurements of time, cost 
and competency Index. For example^ considering the time parameters on 
medical student to resident and medical student to practicing physician, 
the percentage Increases are 19S and 161 respectively. On the cost meas- 
urement the percentage increases are 16| and kk% respectively and for 
competency^ index measurement the percentage increases are 161 and 13l. 
Thus, it appears that the experienced (e.g., residents and practicing) 
physicians are providing the simulated patients with more extensive work- 
ups and taking more time to reach a diagnosis than the medical students 
while gaining somewhat higher competency index scores In the process. 

Finally, there appear to be very small differences between the res- 
idents and the practicing physicians on the time and competency index 
measurements; only a 21 decrease for each from resident to practicing 
physician. However, the cost measurement shows an increase of 25%. Thus, 
percentage differences between residents and practicing physician groups 
appear minimal for all but cost measurement. 

It should be noted the empirical results of these simulated patient- 
physician encounters are primarily of a pilot study feasibility nature 
and further, more controlled experimental Investigations are required. 
However* the results of the continuing computer simulated instructional^ 
and evaluation effort lend support to the growing medical interest In In-^ 
dividual ized Instruction via the computer and also offer potential for 
instructional innovation In other areas through the voice system approach. 

It appears the Voice-'Response System Is an adequate maans for obtain-^ 
ingi performance data on the ability of physicj^ns to obtain and .Interpret 
diagnost ic dita. The affective data Indicates the viability of the voice 
technology as a means of enhancing physician skills. Certainly as the 
technology, both In the quality of the voice and the consistency of phone 
line transmission improves there appears to be preliminary evidence that 
such a voice system can become widespread. 

The simulated encounter does provide clear, safe and measurable al- 
ternatives to a "real-life** behavioral learning/practicing situation 
where the safety, integrity or future repercussions of physician error 
caused by independent actions might be uncontrollable or Irreversible. 
Similar computi assisted encounters in such areas as "Psychologist-client 
or "teacher-Pupil" may prove to be of great future help in the areas of 
Initial Instruction, on-going evaluation, and periodic review by either 
learner or teachers. 



12 



VOICE 

MKDICAL STUDENTS £ A £ E 

6 : 9 i 10 ^13 15 ! TOTAL 



TABLE ONE , • ; ! 

- ! ! 
X - Ave rage | 

N ~ Numbar 
: SD - Stanjard Devlatlc 


1 ' . 

1 
\ 

\ 

i 

! ; 




T^m^ t 75.02 j 

(mlnutas) ; 22*97 7 


^6. 11 
14.19 ^ 5 


12*36 6 


36.56 ^ 39>Q4 | 
6.78 4 8; 27 4i 


24*32 26 


Workup Cost 1 729, 63 


370*35 


241*58 


1619.75 i 704*88 


681.04 


(donari) :358*20 7 


130*05 5 


66*27 6 


783.60 #^156>26 ' 4 

f 
1 


564.06 26 


Hospital Cost j 887. 14 
(dollars) j 210. 61 7 
1 


133.49 5 


150* 00 

109*00 ej 

.. 


■ 1 

360.00 ; 
127.28 4 


517*50 
236.70 4 


463.85 
326.68 26 


Diagnosis 
{% cqrreet) 


.53 3 


1 n 

X . u 

5 


! 

1 0 


.50 j 1.0 

.58 4| 4 


.77 

,43 22. 


Efficiency ^ 

1 


44.20 
8,67 5 " 


D/ m UU 

10.17 5 


1 7 

P^* -L / 

10.53 6 




64*75 
13*84 4 


57,80 

13*16 20 


Proficiency 34,00 

; 9.38 5 


57.60 
17.33 5 


48*50 
10*37 6 




38*25 
8*62 4 


45*10 

14*45 20. 


Errors of ^ 57. AO 
Mission ' 9 91 5 
j 


31.00 
14.30 5 


48*17 
9.41 6 




56.75 
5*12 


47*90 

14*37 20 


Errors of 
Commission 


7*20 
4*79 


10.^20 

. 1.1 .fj 

9.96 3 


2*33 

2.34 6 




4.00 

4*24^ 4 


5.85 

5*99 20 


Compatency 
Index 


15.72 
6.30 5 


34.37 
15.67 5 


32*48 
10.68 6 




25.50 
9.90 


27.37 

12.77 20 



13 



ERIC 



VOICE 

RESIDENTS C A, £ E 



TABLE Th 


^ 6 9 ! 10 


13 




TOTAL 


Avirag-e N ^ Numbar 
-L ^-D ' Standard Deviation ' 








Tlma 
(minutas) 


i 

\ 82»93 7 36*87 IG 
27.35 ; 16,76 


\ ' 35.28 8 ! 54,71 6 
26,003 j 28*06 


i 46,49 8 

! 11.83 

\ 


49.53 39 
27,11 


Workup 
Cost 
(dollars) 


771.54 i 695.89 


473*69 8 
410.94 


2345.67 6 
960.51 


\ 602,68 ^.8 
131.35 


j955,60 39 
|878,63 


Hospital 
Cost 
(dollars) 


1092.86 7 j 342.00 10 
451,28 i 92.95 


180,00 8 
127*28 


405,00 6 
158.46 


393 .75 8 
220,13 


463,85 39 
380,46 


Di€ignosis 
(1 corraet) 


.43 7 1 *90 10 

.53 ^ 


1 ..,875 8 
,35 


,83 6 
,41 


,875 8 
.35 


.795 39 
,409 




50.83 6; 57,56 9 
12.91 ! 18,81 

i 

— — — — _ =_ 


68,14 7 
12,56 




65,14 7 
4,18 


60.55 . 29 
14,53 


Profit 
ciency 


54.17 6| 47.33 9 
12,38 ^ 10.58 


54,86 7 
12,03 


i 

j 


49,71 7 
6.68 


51.14 29 
10,51 


Errors of 
Omis^ 
sion 


37,83 6 
9.81 


i 37,89 9 
\ 13,31 

i 


36.14 7 
6,67 




44,00 7 

/* 


38,93 . 29 
9.91. 


Errors of 
Coamis- 
sion 


7,33 6 
6.02 


1 13,89 9 
10,15 


8,14 7 
9,56 




4,86 7 
2.67 


8,97 29 
8.36 


Compa- 
tency 
Index 


28,42 6 
12,34 


27,86 9 
12.20 


38,20 7 
13-24 




. 32.70 7 
5,22 


31,64 29 
11.44 
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VOICE 
PRACTICING 
PHYSICIANS 

6 

r ^ 

r4fl£ff ra^^E K - Average 



Time 

(ml nutes) 



SD 



87.55 
50.03 



10 



N - Number 
Standard De v i at j^n 



38.47 10 
14.79 



Workup Cost 2048.89 9 
(dollars) 2752.15 



Hospital 
Coat 

(dollars) 



1000,00 
222.49 



Diagnosis 
(I eorract) 



Eff Icleacy 



Proficiency 



.33 9 
.500 



344.85 10 
187.23 



342.00 10 
125.86 



56.22 9 
16.14 



37.89 9 
16.18 



.800 10 
.42 



-71.800 10 
17.94 



Errors of 
Omission 



54,89 9 

13.65.,. 



54.00 10 
13.43 



29,55 U 
12.75 



379.86 U 
215.42 
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180.00 11 
113.84 



.909 n 
.301 



70.909 11 
13,43 



54.64 11 
11.42 



39.50 10 
14.29 



39.909 11 
11.26 
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3|.29 
10.93 



927.86 
451.75 



47.99 36 
35.26 



878.73 36 
1514.88 



810.00 6 

98.59 



.166. 6 
.408 



54.50 
16.56 



535.00 36 
373.22 



.611 36 
.494 



26.17 
10.98 



67.33 
10*33 



64.75 36 
17.20 



45.53 36 
16.93 



48.11 36 
16.25 



Errors of 
Comlsslon 



6.22 
9.19 



5.80 10 
5.51 



4.18 

3.95 
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5.33 6 
4.76 



5.33 36 
5.96 



Competency. 



22.10 
12.43 



39.095 10 
15.43 



39.45 
11.499 



11 



30.95 36 
15.85 
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TABLE FOUR 



Summary Rate for the Vofce Response System(for cases 6-15) 







Time 


Cost 


CI 




N 


X 


SD 


X 


SD 


X 


SD 


Medical Students 


20 


it1.26 


Z2.5k 


975.98 


6i»5.00 


27.37 


12.77 


Residents 


29 


^9.18 


rim 


1130.01 


886.00 


31.6i| 


W.kk 


Practicing Physicians 


36 


^7.99 


35.26 


1^13.73 

- 


172it.00 


30.95 


15.85 
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APPEUDIX 3; Sample Data Summary 



Ft)3 
•560, 



AM 

00 



RfcCOHD 255 
USKRs 20 

USKH bsGAN' AT ViOV PM AND ENDED AT 75 43 PM 

cosrs* wonKUPt $ 

UAY3 IN HOSPITALi 5 

HCJSPI TALI NATION $90/DAY 450,00 
TOTAL? S 1010.00 

CORHEiUr UIACNOSIS? YES 

b' VALUATION-. YES 



SCRIPT NUMBER 900^ 
RUN DATES APR 23 1976 
ELAPSED TIME OH Os36?34 



TEST OR DX 



TIME TEST OR TIME RESULT RECEIVED 



TX OF?DERED 



OR DX MADE 



HISTORY 


10 


DAY 


1 


I2?00 


DAY 


1 


Utj2 


PHYSICAL EXAMINATION 


13 


DAY 


1 


12100 


■ DAY 


1 


I 2? 36 


STAT COMPLETg BLOOD COUNT 


12 24 


DAY 


\ 


1 2s00 


DAY 


1 


!2?30 


STAT rtHITE BLOOD CELL DIFFEBEMTIALI 257 


DAY 


1 1 2 i 00 




DAY 


I 1 3? 00 


STAT ELECTROCAnDIOORAM 


1036 


DAY 


1 


I2IO0 


DAY 


1 


12? 12 


STAT CHEST X-RAY 


1061 


DAY 


1 


12I00 


DAY 


I ■ 


12? 30 


STAT CREATINE PHOSPHOiCINASE 


1284 


DAY 


1 


I2;00 


DAY 


1 


I4?00 


SMA 1 2 


309 


DAY 


1 


12 -.00 


DAY 


3 


17*00 


STAT URINE ANALYSIS 


1 166 


DAY 


1 


I2i00 


DAY 


1 


12130 


STAT ELECTROLYTES 


1290 


DAY 


I 


I2;00 


DAY 


1 


1 2 ? 30 


ARTERIAL BLOOD CASES 


263 


DAY 


1 


12*00 


DAY 


1 


17100 


STAT FLATPLATE OF ABDOMEN 


1067 


DAY 


1 


I2S00 


DAY 


1 


12? 30 


STAT LUNG SCAN 


1087 


DAY 


1 


12 = 36 


DAY 


1 


13? 36 


LUPUS ERrfHEMAlOSUS PREPARATI0N2 1 2 


DAY 


I 


I2s36 


DAY 


3 


I7?00 


GASTROrfiTESTINAL SERIES 


63 


DAY 


I 


13? 00 


DAY 


2 


I4?00 


ERYTHROCYrE SEUI MENTATION RATE 


228 


DAY 


I 


I 3 : 00 


DAY 


2 


I7?00 


SURGICAL CONSULTATION 


117 


DAY 


I 


I3?00 


DAY 


I 


14100 


TRIOLYCERIDSS 


339 


DAY 


I 


I3?00 


DAY 


5 


1 7 ? 00 


UIOPSY 


20 


DAY 


2 


I4t00 


DAY 


4 


03? 00 


SERUM IMMUNOELECTROPHORESIS 


296 


DAY 


3 


17100 


"DAY 


5 


17100 


**** 6000 




* 


* 




DAY 


5 


09? 00 CORR 



EFFICIENCY 45 

PHOFICIENCl' 43 

ERRORS OF OMMISSION 30 

ERRORS OF COMf.ttSSION 26 

COMPETENCY IIIDEX 19.3d 
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Replies receivid froni 7 thlrd'yeir medieal students and \k nisdlci! residents tistid. 
(Hedica! reiidenti hive been broken down into flrit, second ind third year,) 



3rd yr. Hed lit yr. Eii. 2nd yp. Res. 3rd yrj Res. 

%/l m no ^ Hi no ^ ^ no ^ M 1. 

Were you ible to underitind the spitch? 0 J 0 0 5 0 0 I 0 0 3 0 
Old you find this prebliriolvlng 

exarcise rsilistlc? 2' J 0 1 11 1 1 3 1 0 2 
Do you fill this is a reasonibli 

method of caiiinunicition? 0 7 0 0 i| 1 15 0 12 0 
Do you fee] this could serve a 

continuing education function? 0 7 0 0^1 0 I 2 012 
Do you fill this is in iffectivs 

means of sslf^sviluitlon? ^ 3 0 0 3 2 1 1 2 01 

Did you incounteniiechinici! difficulty? 0 5 2 0 3 2 0 5 1 0 3 0 

Did you enjoy worlcing with thiiystifii? 0 1 0 0 'i 1 Q S 0 0 .1 2 



22 - within limits 23 



ERIC 



APPENDIX V 



VOICE SOFTWARE DOCUMENTATION 



24 



Votrax Synthetic Voice Rosponse on PDP-IS System 



January 16, 1976 

General Design and List of Routines 

The program for voice response on the F0P-=15 ' consists of four routines 
used to speak English and numeric data. These are all called by one common . 
entry point and will convert English text to spoken outputs For faster 
access vhen the construction of the output is knoiJn in advance , string of 
pointers to the glohal nodes mmy replace English words. 

In addition to these four major routines there are nine utility routines. 
These are usad for ^building the dictionary, finding words in th© dictionary, 
daletlng words, transferring words, and listing the entire contents of the 
diationary* ^ 

The major programs are as follows s ^ . 

1) VSP ^ drivar program for speaking subroutines 

2) VQZ ^ spaak teKt output or pointer output , _ 

3) VON « speak numeric output or time output 

4) VSS ^ repeat last spoken output 



The utility programs arei . • 

1) YAM* - move a word into an additional or alternate spelling 

2) VAK - remove a word from the dictionary 

3) CVZ " dictionary progrto for use yitth PDP-8 dictionary routine 

to allow entering and editing of dictionary 

4) VAG ^ find a word in the dictionary and give phonemef^ and octal 

squivalent for the English 

5) VAD ^ utility program used by VAC and CVE to locate words in global 

6) VLE ^ transfer words from dictionary into global CH for sorting and 

printing 

7) VLT ^ Delete duplicates from list of "needed" dictionary words 

8) ^0 ^ list all words in dictionary 

9) VLQ list all words not found in the dictionary which appeared in 

ti^t to be spoken 



VotraK Synthetic Voice Response on PDP-IS SysCam 



TITLBl _VSP 

PURPOSEi Major spealcing driver program 
PROGRAi'l>mRi Steven Entlna . 
DATE: JanuaBy 16, 1976 

This program requires data in variable on what to spead. XX is a etrlng 
valued variable that may contain English wordsi numerie data, or ^ times of 
the day* These are separated by one space. The routing can also accept 
pDinters to a global containing the octal* This is flagged by replacing 
a word that would normally appear in XX by the two pointers to 'the global 
precaded by a ^ sign. Hencep rather than the word "hello'^ one could replace 
it with "J3i4,1290". .-.Iffille this looks unreadable it will execute much fasteri. 
Pointers and English may be interspersed at will. In addition to XX other 
variable may be defined* : . ' 

1) ANS ^ if defined, an answer Is desired from the user and the answer is 
to be placed in the variable whose name la in ANS- Hence ^ ANS should 
be a string of 1*2, or 3 letters. , ^ 

2) END - if defined the output should be spoken Immediately and the buffer 
in the Votrax Synthesizer should be cleared^ 

3) DBG ^ if defined a user is debugging a program from a terminal* English, 
a^ well as the phonems^ will appear as output* This cannot be used with 
the current PDP-S programs. It is strictly for debugging purposes. If 
DBG ^ "QUI" only English (no phonems) will be printed. 

In addition to theses the routine also uses variables BB, BBB, BC and many 
variables preceded by a % sign* The calllng^programs should not uaa these 
variables* 

Prograsi VSP is used to signal the end of the string to the speaking routines 
and to prompt the PDP^S to wait for an answer if one is needed, of empty the 
buffer if that has been specified* It calls VOZ for al] actual speaking. 
This routirie also checks if user's Input has been a ^'^7" to signal repeating^ 
If this is the case routine calls program VSS to repeat a messagep The - 
Routine always kills variable M^B and GOT. ^ . r 



pROGR^s mm% VOZ vss 

GLOBALS VSm% CVS 



VotraK Synthetic Voice Response on System 



TITIiE^ VOZ 

PURPOSE: This program actually will speak the tmnt in variable XX 
PROGRAMr^ERr Steven Entine 

DATEi Janua^ 16,i976 . -/* 

The program will break KX into individual v;ordS/ numbers^ or pointers. 
These are separated by a spacep For each ot these i VOZ will determine 
if it is a word^ in which casa it will find it in the dictionary and 
speak it; a number-* in which case program VON is called to speak it; a 
time^ which also uses VONs or a direct pointer into the global* 

In case of words ^ the word is hashed according to the dictionary storage 
foinnat^ and an attempt is made to find it in global WW. This global 
contains a list of all global words in the dictionary* If it is 
suacessfully founds the correspionding node of global WO has the necess- 
ary phonemes to speak* These are spoken and are also stored in global 
CVS for recall if the user requests the message to be repeated* If the 
word is not found in global VVW^ then nothing is spoken* However^ it is 
tagged and logged in global ZOT for later use* A listing of all %7ords 
in ^OT can" be printed out* A decision can be made if these should indee 
be added to the dictiona^* 

In the case of direct pointers into the global^ the appropriate node of 
global WO is sent to the PDP-8 for speaking and also logged into CVS 
for repeating* No search in the dictionary is necessary nor are any 
checks made to assure that this word really does exist* It is the user'r 
responsibility when using pointers to be sure that the node is in fact 
in the dictionary* 

This routine and routine VON use variable BB to keep track of the .iumber 
of phonemes that have been sent out in this buffer. If this ninnber is 
greater than the threshold (usually 80) , a signal is sent to th^ PDP--8 
to speak this buffer and propare for input for. the next buffer* The 
Votrax cannot handle more than one buffer of speach at a time and blockin 
is taken care of in routines VOZ and VON* If variable DBG is defined the 
this routine will print out the English as well as^ the phonemes* When 
it is sending messages to the PDP--8 to clear a buffer this is alt shown 
at Uie terminal* 

PROGRAMS USED^ VON 

GLOBALS USED I CVS, WO^ WW^ ZOT 
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Votrnx Synthetic Voice RGsponse on System 
TITLE i^VO^ 

PURPOSE: Speak numbers and time 
PROGRAMMER: Steven Entine 
DATEi January 16^1976 

This program is callefl by VOZ to speak numeric data and times of 
the day^ It will automatically translate numbers of the form *'1234-' 
and speak them as "one thousand t\^o hundred thirty four"* It can 
do this for nmnbers up to nine hundred ninety nine thousand^ ninB 
hundred and ninety nine point ninety nine* For numbers greater than 
one inilliort the digits are spoken one at a time# For digits to the 
right of the decimal pointi if there are three or fewer they are 
spoken one digit tat a time* Leading zeros are always spoken for nuit! 
bers to the right of the decimal points 

Times are signaled by having a colon in them* In this case all num-* 
bers are always spoken so that 12:06 will speak as twelve 0 six* 
The variable BB is used to be compatible v;ith VOZ in assuring that 
the Votrax buffer is not over filled* It also places all variables 
spoken into global CVS for repeating* If variable DBG is defined | 
then the nutttoers that are spoken are also printed on the terminal* ^ 

PROGRAMS USED I None - ; ' 

^ t ■ 

GLOBALS USED^ " VO, CVS ' . . 
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Votrax Synthetic Voice Response on PDP-Is System 
TITLE: VSS 

tURPOSE: Repeat last message spoken 
PROGRAlOffiR! Steven Entine 
DATE I Januaify 16 ^ 1976 

VSS Called only When a user responds to ^prompt for input with "*7" 
It will repeat ail words spoken by the Votmx up to the user's previous . 
input. Ic does this by using global CVS^ere the phonems have been 
teraporarlly stored, rather than reconstri^lng from the basic dictionary 
For this reason repeated raessages should be%eard much quicker than 
messages the first time. There should be ^^apparent lag In response ttae. 

PROGRAMS USED I. None 
GLOBALS tJSEO.' CVS 



Storing V/ords for the Voice Response System 



V/e need a program to allow permanent disk storage for nev; v/ords. These v/jll 
be used by the Votrax voice response system. Each word is stored In three formsi 
a. The usual english spelling 

b* A list of 'phonemes'. This will be constructed using the PDP^S dictionary 
program. (This dictionary program runs independently of the PDP-15) 
In an *octaP format for use by the votrax synthesizer. 

There are utility routines which will translate phonemes into the octal and store 
the information in the dictionary. What is needed Is a program to allow a user to type 
in the english word and the corresponding phonemes^ edit them as needed^ verify 
that they are correct and should be stored permanentlys and then call eKistIng 
programs to do the translation and actual storage* , 

An example of the threa types of storage is: 

a, FIELD 

b, IpaO 2f 2ie 1 iy iuh3 IL id .... 

c, HCMHOLJIJCIHIN ' 

You will only hmvm to be concerned v;lth types a and b* I have a separate list of 
legal phonemes* Each is a number (from 1^3) followed by 1"3 alphanumeric characters* 
Your program should be done in password FPS, Its ID should be VAH, You should not 
use any globals* 

The user should be allowed to enter the english and the phonemes". They should be able 
to re-enter the english and edit (by deletings addingpOr replacing a single phoneme) 
the corresponding phonemes* They should then be able to certify the combination for stc 
or declare the entry as garbage* 

Once you have a certified entryp you should place the english into variable 'Y' and 
the phonemes Into variable •X'* SPACES and punctuation are crucial. The english can 
NOT have ANY spaces* If someone wants to store 'GOOD MORNING% It should be dona as 
2 separate entries* Phonemes should have EXACTLY one space separating them* 
ThuSj in the above axample^ you should have coda that would accomplish: 

Y^^FIELD" X^^lpaO 2f 2le liy Iuh3 II Id*' 
Then CALL VAE. This will coda the phonemes into the octal and store the. word In the 
permanant voice dictionary. If the word already exists^ the user will be given a warnir 
massaga and asked If this entry Is to replace the old one. If an Illegal phonama is 
passed to this routine^ a message will be given and the word will be Ignored* 

To check if a word was properly stored, you can use program VAC* This will ask you for 
a word* If It Is not In the dictionaryp you will be so informed* If It Is, It will 
print the Indices In the globalp the piiQiremes, and the octal. There are also programs 
to cfelete a word or to create a duplicate (the later Is useful for abbreviations 
or homonyms*) 
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Storing and changing voice dictionury words In the PUP-15 

Wa need a program to allow tht* addition find con act ion of viords in the voice vocal 
It should be progrum CVW in password FPS. (if you need more MD's, use tVU,CVR) 

Each v/ord Is ^ pred In three forms ^ 

a. The usual english spelling 

b. PhDnemes used by people to create and edit the spoken v/ord* 

c. Phonemes used by the PDP-'S and Votrsx to actually speak the word* 

The user will key in versions a and b. Using version b and the folloiwing algoriti 
the program must conipute version 'c'* 

EKamplei The english word FIELD (type ."aO has phonemes ,^ , ^ 

IpaO 2f Zie liy iuh3 IL id (type'b') ' 

and the votraK uses HCMHOLJ I JCIHIN 

Note the fonowingi The phonemes used by people are ALWAYS I number fbllowed by 

one to three other characters. Each of these corresponds to TWO letters In^ 
the phoneme used by the votrax. There 'are 256 possible phonemes. 

I have a table listing the corrspondlng OCTAL numeric value for each basic phonc-.m 
By a basic phoneme, I mean the characters that follow the number. The number !s ft 

an *occent" flag* 

Examplei . P&o is OJg ' * 

f is . ^ — 

ie is 7hg ' ' 

To these basic values^ v/e use the foUowing offsets for the accent. 
For a 'r, proceed the value v/ith 2. For a '2', preceed the value with 3. For a 
■3', preceed the value with a '0'* 

Thus IpaO is 203g; 2f is 335g; 2ie Is Zlh^i , . 

3ie would be 07^g- 

This will always represent 8 binary bits. It will be 3 octal numbers , but the firs 

will never be h or inore* Separate this into 2 numbers of h bits each. 

Thus Ipao is 203© or as a binary number it is lOOOOOllg 

Separated into two numbers we geti 1000 0011 

Convert these two binary numbers into a decimal number 63* 

Now* convert these Into the corresponding letter of the alphabetic UC, 

Example 2: f^SSg so 2f Is 3353-1 101 1 101 -llOl 1101-13,0 13,o-MM 

Example 3^ ia-74g ;2le«374g-l 1 11 1100-1 1 1 1 1100^ 15 12- OL 
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